-
Notifications
You must be signed in to change notification settings - Fork 27
smarter action shuffler #39
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
1a0566f to
0b4e0f6
Compare
| implementation("com.atlassian.performance.tools:concurrency:[1.0.0,2.0.0)") | ||
| implementation("com.atlassian.performance:selenium-js:[1.0.0,2.0.0)") | ||
|
|
||
| runtimeOnly("org.jetbrains.kotlin:kotlin-reflect:${kotlinVersion}") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Uh oh, reflection = danger zone. We're bypassing a lot of compilation-time guarantees.
| } | ||
|
|
||
| companion object { | ||
| fun createRandomisedScenario(seededRandom: SeededRandom, actionProportions: Map<Action, Int>, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's the problem we're trying to solve here?
Do skipped actions malform the ActionMetric.label proportions too much? Our initial workaround was to run the test for a longer time so they get averaged out.
If that is not effective enough, then we can avoid action skips by using a self-loading Memory, e.g. pass this to an action during Memory.recall:
fun recall(): String {
if (issueKeys.isEmpty()) {
SearchJqlAction(jira, throwawayActionMeter, IssueJqlMemory(), this).run()
}
}This way you can avoid skips with local logic rather than a global order-controlling scheme.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. I want to preserve the requested proportions. During our runs, ~1500 view issue calls are skipped. This looks like a minimally invasive way of achieving this, without altering the way scenarios are created.
The problem with hiding the query inside a different action is that you lose control over proportions - if there are permission issues, the JQL action can be executed as often as view issue, which we don't want. If we stick to sorting, everything is transparent and there are no surprises hiding in the implicit initialisation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The same problem affects ViewBoard, btw.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's try a different question: is there any benefit to keeping a fully-randomised shuffle in the scenario?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🤔
I'll come back to this next week
0b4e0f6 to
c9aa5a8
Compare
4d10d91 to
d9a2e73
Compare
d9a2e73 to
7a98551
Compare
No description provided.